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Abstract — Generalized orthogonal matching pursuit (gOMP) 
is an extension of orthogonal matching pursuit (OMP) algorithm 
designed to improve the recovery performance of sparse sig- 
nals. In this paper, we provide a new analysis for the gOMP 
algorithm for both noiseless and noisy scenarios. We show that 
if the measurement matrix 3> 6 R mx ™ satisfies the restricted 
isometry property (RIP) with 5 7 k+n-i < 0.0231, then gOMP 
can perfectly recover any if-sparse signal x 6 TV 1 from the 
measurements y = <&x within \^jf] iterations (N is the number 
of indices chosen in each iteration). We also show that if <l? 
satisfies the RIP with S 11k +n-i < 0.0627, then gOMP can 
perform a stable recovery of A"-sparse signal x from the noisy 
measurements y = <l?x+v within ["^Fl iterations. For Gaussian 
random measurements, the results indicate that the required 
measurement size is m — 0(K log(-^)), which is much smaller 
than the existing result m = 0{K log(-^)). 

Index Terms — Compressive sensing (CS), sparse recovery, sta- 
ble recovery, generalized orthogonal matching pursuit (gOMP), 
restricted isometry property (RIP) 



I. Introduction 

Orthogonal matching pursuit (OMP) is a greedy algorithm 
widely used for solving sparse recovery problems fll]-|J6]. The 
goal of OMP is to recover a X-sparse vector x € lZ n from 
the measurements 

y = *x (l) 

where 4> s 7^ mx ™ j s a measurement matrix. In each iteration, 
OMP estimates the support (positions of nonzero elements) of 
x by adding an index of the column of 3? which is maximally 
correlated with the residual. The vestige of columns in the 
estimated support is eliminated from the measurements y, 
yielding an updated residual for the next iteration. While the 
number of iterations of OMP is usually set to the sparsity 
level of the underlying signal to be recovered, there has been 
some attempts to relax this constraint to enhance the recovery 
performance. In one direction, an approach allowing more 
iterations than the sparsity level has been suggested Q-JTO). 
In another direction, an algorithm selecting multiple indices 
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(No. 2012R1A2A2A01047510). 



TABLE I 
gOMP Algorithm 



Input: 


measurement matrix <I> G R mx ™, 




measurements y 6 M m , 




sparsity level K, 




number of indices for each selection N . 


Initialize: 


iteration count k = 0, 




estimated list T° = 0, 




residual vector r° = y. 


While 


||r fc ||2 > e and k < m/N do 




k = k + l. 




A k = argmax A .| A | =JV ||(#'r fe_1 )A||i. (Identification) 




T k = T k ~ 1 U A k . (Augmentation) 




x fe = argmin supp(u)=T fc ||y - *u|| 2 . (Estimation) 




r k = y — $x ft . (Residual Update) 


End 




Output: 


the estimated signal x fe . 



in each iteration, referred to as generalized OMP (gOMP) 
fTO . OMMP Q3, fUl or KOMP Q3, has been proposed. 
Since it is possible to choose more than one support index 
in each iteration, gOMP is in general terminated in less than 
K iterations. Also, it has been empirically shown that gOMP 
is better than OMP algorithm in recovery performance and 
computational cost fTTi , |14|. 

In analyzing the theoretical performance of gOMP, a prop- 
erty so called resUicted isometry property (RIP) has been 
popularly used fTO . Ifl2l . Ifl4l . A measurement matrix €> is 
said to satisfy the RIP of order K if there exists a constant 5 
such that [151 



(l-«J)||x||»<||#x||l<(l + *)||x||i 



Mia 



(2) 



for any iiT-sparse vector x. In particular, the minimum of all 
constants 5 satisfying (ffji is called the isometry constant 8k- 
It has been shown that the perfect recovery of any if-sparse 
signal via gOMP algorithm is guaranteed under ifTTI 



Snk < 



K + 3VN' 



(3) 



For Gaussian random measurements, this result implies that 
the required size of measurements is ||T6l 



O K 2 log 



K 



(4) 



While the perfect recovery analysis is possible for noiseless 
scenario, such is not possible for noisy scenario where the 
measurements are contaminated by a noise vector v as 



y = *x 



(5) 



To analyze the recovery performance in this scenario, the £2- 
norm of recovery distortion has been commonly employed. In 
IfTTI . it has been shown that under 



Snk < 



N 



and S 



N(K+1) 



gOMP generates an estimate x A such that 



<1, 



K 



x|| 2 < CidlvlU 



(6) 



(7) 



where C K = 0(y/K). 

The main purpose of this paper is to provide improved 
recovery bounds of gOMP algorithm in both noiseless and 
noisy scenarios. The primary contributions of this paper are 
twofold: 

1) In noiseless scenario, we show that if the measurement 
matrix $ satisfies the RIP with 



HK+N-l 



< 0.0231, 



(8) 



then gOMP perfectly recovers any if-sparse signal 
within r^wH iterations. 
2) In noisy scenario, we show that under 



5iik+n-i < 0.0627, 



(9) 



gOMP can perform a stable recovery of X-sparse signal 
x from the noisy measurements y = <I>x + v within 
r^TT"! i tera ti ons with the recovery distortion satisfying 



rlOKi 
r N 



x|| 2 <C||v||a 



(10) 



where C is a constant. 
In comparison to our previous work in Hill , our new results 
are distinct and important in two aspects. Firstly, in contrast 
to the previous recovery bounds in (|3) and ©, which are 
expressed as monotonically decreasing functions of K and 
hence obviously vanish at large K, the proposed bounds in (0 
and (O are constants. When Gaussian random measurements 
are employed, the proposed bounds imply that the required 
measurement size is [16| 



O 



(**(£ 



(11) 



which is clearly better than the previous result m = 
O (K 2 log (j|)) fin . Secondly and perhaps more impor- 
tantly, while our previous work demonstrates that the recon- 
struction error (i.e., the ^-norm of recovery distortion) in 
noisy scenario depends linearly on Va||v||2 IfTTI . our new 
result suggests that the reconstruction error is upper bounded 
by a constant multiple of ||v||2, which ensures the stability of 
gOMP algorithm under noisy scenario. 

The remainder of the paper is organized as follows. In 
Section [HI we introduce lemmas and propositions that are 
useful in our analysis. In Section Hill we provide the recovery 
bound analysis of gOMP algorithm in both the noiseless and 



noisy scenarios. In Section HVl and IVl we provide the recovery 
condition analysis of the gOMP in the noiseless and noisy 
scenario, respectively, and conclude the paper in Section |VT] 
We briefly summarize notations used in this paper. For a 
vector x e 7Z n , T = supp(x) — {i\xi 7^ 0} represents the 
set of its non-zero positions. SI = {1, ■ ■ • , n}. For a set S C 
£1, \S\ denotes the cardinality of S. T\S is the set of all 
elements contained in T but not in S. <f>s G 7?. mx ' s ' is a 
submatrix of 3? that only contains columns indexed by S. 
<fr' s means the transpose of the matrix <&g. X5 € T?.' 5 ' is an 
vector which equals x for elements indexed by S. If <&s is 
full column rank, then &' s = ($' s $s) _1 *' s is the pseudo- 
inverse of $5. span($s) stands for the span of columns in 
$5. Pg = ^s^s is the projection onto span(&s)- Pg = 
I — P5 is the projection onto the orthogonal complement of 
span(&s)- 

II. Preliminaries 

In this section, we provide lemmas and propositions that will 
be used throughout the paper. Let T k = T\T k , then T k is the 
set of remaining (unselected) support indices after k iterations 
of gOMP (see Fig. |l(a)[ ). Here and in the rest of the the paper, 
we assume without loss of generality that T fc = {1, • ■ • , |r fc |}. 
Then it is clear that < |r fe | < K. For example, if k = 0, 
then T k = and |r fc | = \T\ = K. Whereas if T k D T, then 
r fc = and |r fc | = 0. Also, for notational convenience we 
assume that {^i}i=i,2,-- ,|r fc | are arranged in descending order 
of their magnitudes, (i.e., \x\\ > |cc2 1 > • • ■ > |^|r fc |D- Now, 
we define the subset T k of T k as 



0. 



r . 



{M,-..^- 1 ^} 7-=l,2,...,[log 2 Vl> 



|r fc |- 



[log 2 



m - 

N 



(12) 
See Fig. |l(b)| for the illustration of T k . Note that the last 

set T k ,- k , does not necessarily have 2r iog 2 ~n^1 N 

elements IJ 

For a given set T k and a constant /j, > 2, let L € 

ipfc I 
{1, 2, • • • , |~log 2 fj\ + 1} be the minimum positive integer 

satisfyin 



x r fc \rg II2 < Mll x r fc \r^ll2i 

i|2 ^ 11 ||2 

l x r fc \rf II2 < Mll x r fc \r|ll2j 



(13) 
(14) 



X rfc \ 



v r*\rj,_ 2 ll2 < Mll x rfc\r*_Jl2! 
I x r*\r*_jl2 > Mll x rfc\r* II2- 



(15) 
/"Il x rfc\r*ll2- (16) 

Then, we can easily check the following inequalities. 

-ifc 

L 



Lemma 2.1: T k and T k satisfy 



1) 



|r£| < |r fe | 



l 2 



= {!,■■ 



, 6Af+l}, then rg = 0, rf = {1, • • ■ , N}, 
■ ■ , 47V}, andrj = {1, • • • , 6/V+l} = T k 



'For example, if r fc 

.={1. 
(|rJ|=6JV + l <8N) 

2 We note that L is a function of fi and k. Here we use L instead of L^^ 
for notational convenience. 



Q 



T (support of x) 




(estimated support) 



(a) Set diagram of T, T k , and V k 




Fig. 1. 



2) 
3) 



2 -""3 

(b) Illustration of indices in T k 
Illustration of sets T, T k , and Y k . 



| r fc| < 2 L ~ l N, 



1 nog 2 ^i + i--i 



ir fe | > 2 



L-2 



TV, 



The following lemma provides an upper bound of 1 1 jr fc 1 1 2 - 

Lemma 2.2: The residual r k of gOMP satisfies 

||r fc ||2<||* rtXrfc +v||2. 

Proof: See Appendix lAl ■ 

The following proposition provides a lower bound of 

llr'lll - ll r ' +1 |ll ( l > k ) for gOMP algorithm. 

Proposition 2.3: Let x G TV 1 be any X-sparse vector 
supported on T, $ e j^mxn ^ a measurement matrix, and 
y = 4>x + v be the noisy measurements. Then for a given T k 
and any integer I > k, the residual of gOMP satisfies 



r'll 2 - llr 
1 9 J- 



Z+l||2 



1-<S 



> 



|r*ur'| 



„Z 1 1 2 



r^i(i+M 



> r k\ r kX r k\ r k + v|| 2 ) 



o,i > ...,fio ga jyi 



1. 



where r 

Proof: See Appendix iBl 

Proposition 12 .41 can be viewed as a generalized version of ifTUl 
Proposition 3.4]. From this proposition, one can further obtain 
the following proposition, which will play a key role in our 
analysis. 

Proposition 2.4: For any integer I' > I (> fc) and r € 

{0, 1, • • • , |~log 2 ■l^i] +1}, the residual r r of gOMP satisfies 



J'||2 



<c T . 



w 



„i ||2 



*pfc\ pfcXpfc\ y-k H~ V|| 



where 



C T j « = exp - 



(i - /)(1 - <5| r fc UT i'-i|) 



r^i(i+*. 



(17) 



N) 



Proof: See Appendix ICl 



III. Recovery of Sparse Signals using gOMP 

In this section, we study the performance of gOMP algo- 
rithm in recovering sparse signals in both the noiseless and 
noisy scenarios. We first provide a condition under which 
gOMP perfectly recovers sparse signals in the noiseless sce- 
nario. In this scenario, gOMP can recover any if-sparse signal 
x accurately as long as all support indices of the signal x 
are chosen. When the iteration loop of gOMP is finished, of 
course, it is possible that the estimated support T' contains 
indices not in T. Even in this case, the final result is unaffected 
and the original signal x can be perfectly recovered as long 
as T C T f because 



rf 



arg 



*x|| 



supp(x)—Tf 



and 



(x/) 



Tf 



rfit „ _ flst 



At 



*L/y = *^ f #TX T = *^$ T /x T/ = x T 



Clearly, it is of importance to find out the iteration number / 
satisfying T C TA 

Theorem 3.1 (Exact recovery condition for gOMP): 
Let x <E lZ n be any if-sparse vector supported on T and 
<fr <G TZ mxn be the measurement matrix. Then for any positive 
k, which is a multiple of h, and integer s = (4k+1)K+N— 1, 

there exists f < k + \^p-~\ such that T C T f , provided 
that 



1 + S N 4(1 + 6 K ) 

■ In — < k. 



1-6. 



1 



(18) 



One can interpret from Theorem l3.1l that after performing k 
iterations of gOMP, the remaining support indices in T k will be 
selected within [ K L ' ~\ additional iterations. In particular, by 
applying k = 0, one can easily observe that T k = T\T° = T 
and / < r^jjpl, which implies that gOMP selects all support 
indices of x within [-^r-] iterations. 

Corollary 3.2: Let x £ 1Z" be any if-sparse vector and 
let $ G j^mxn k e a measurement matrix. Then under 
87K+N-1 < 0.0231, gOMP perfectly recovers x from the 
measurements y = $x within \^-~\ iterations. 

Proof: Applying k = and k = 1.5 in Theorem 13.11 
r fc = T, / < [^], mds = 7K + N-l. Since S N < S s and 
8k < $s in this setting, one can verify that ( fTBl holds under 
Stk+n-i < 0.0231. ■ 

Next, we consider the recovery performance of gOMP in 
the noisy scenario where the measurements are expressed as 
y = <I?x + v. Due to the addition of noise v, the perfect 
reconstruction of x cannot be guaranteed, and hence, we use 
the ^2-norm of recovery distortion as the performance measure. 

Theorem 3.3 (Stable recovery under measurement noise): 
Let x G 7Z n be any iT-sparse vector supported on T, 



<i> G TZ mxn be a measurement matrix, and y = 3>x + v be 
the noisy measurements. Then for any positive k, which is 
a multiple of i, and integer s = (4k + l)K + N — 1, there 

exists f < k + |~ 4K j^ [ ] and constant C and C" such that 



and 
provided that 



\\* f h <C'l|v|| 2 
^-x|| 2 <C||v|| 2) 



1 + far. 8(1 + <5k) ^ 

■ In 1 — < K. 



1-S m 



l-5 s 



(19) 



(20) 



(21) 



One can interpret from Theorem 13.31 that after performing 
k iterations, gOMP will produce a stable recovery of x within 
[ K L 1 additional iterations. In particular, by applying k = 0, 
one can obtain that T k = T\T° = T and / < |~^~|, which 
implies that the stable recovery of any A-sparse signal x is 
guaranteed within |~ j|f^1 iterations of gOMP. 

Corollary 3.4: Let x £ lZ n be any A-sparse vector, $ € 
■gmxn ^ e a measurement matrix, and y = 4>x + v be the 
noisy measurements. Then under 5uk+n-i < 0.0627, gOMP 
satisfies 

Ux^l -x|| 2 <C||v|| 2 

where C is a constant. 

Proof: Applying k = and k = 2.5 in Theorem 13.31 we 



haver fc =T, fc+r^^f-H - [±j^],mds = llK + N-l. 
In this case, Sn < S s and 5k < S s , and hence one can easily 
show that (EU holds under 5u K +N-i < 0.0627. ■ 

Remark 1 (Stability): We have shown that under the RIP, 
gOMP can perform a stable recovery of sparse signals under 
measurement noise. Although we are primarily interested in 
recovering sparse signals, the result can readily be extended 
to more general scenarios where the underlying signal is not 
exactly A-sparse. In essence, our result closes the gap between 
the theoretical performance of gOMP and other stable recovery 
approaches (e.g., the BP 07), Q8), ROMP Q9), and CoSaMP 
EDI). 

Remark 2 (Measurement Size): It is well known that a 
random measurement matrix $ e -j^mxn wn j cn nas j j j 
entries with Gaussian distribution N(Q, — ) obeys the RIP with 
5k < £ with overwhelming probability if |[T6l 



= 



Klog% 



When the bound 6 



NK < 



VW 



, — .,^y H2 is applied, one can 
check that the gOMP can recover any X-sparse signal given 
to = O (K 2 log (jl)) Gaussian random measurements of 
that signal. Whereas, using the proposed bound 5tk+n-i < 
0.0231, one can obtain to = O (-/flog (#)), which is better 
than the previous result, in particular for large K. 



Remark 3 (Recovery Bounds): In Corollary 13. 2| and [3. 41 we 
have used k = 1.5 and k — 2.5 to obtain recovery bounds for 
gOMP in noiseless and noisy scenario, respectively. For sure, 
when different k is used, one can obtain different recovery 



-A — Noiseless scenario 
-Q — Noisy scenario 




Fig. 2. Recovery bounds of gOMP where s = (4k + 1)K + N — 1. 



bounds. Fig. |2] displays a series of recovery bounds (i.e., the 
upper bounds of 5 S ) as a function of k for the noiseless and 
noisy scenarios. We observe that the overall behaviors for two 
scenarios are analogous. For both cases, the upper bound of 
<5 S increases monotonically with k. 

Remark 4 (OMP): OMP is a special case of gOMP for 
N = 1. Although the performance of OMP is not better than 
gOMP, it has been shown that the gap can be reduced when 
OMP runs more than K iterations. In particular, when the 
number of indices chosen by OMP is similar to that of gOMP, 
recovery performances of the two become comparablejj In 
fact, there have been some studies on the recovery performance 
of OMP when it runs more than K iterations lT7l- lfT0l . In [8|, 
Zhang showed that under 83m < 0.3333, the stable recovery 
of A"-sparse signals can be achieved in 30A iterations. It is 
interesting to mention that we can also derive recovery bounds 
for OMP running more than K iterations using Theorem 
13.11 and 13.31 Indeed, by applying N = 1 in Theorem 13.31 
s = (4k + l)K and ( 1211 becomes 



1-6, 8(1 + 5 K ) . 
=- m — — < K. 



(22) 



Using relaxations 5\ < S s and 5k < 8 S to d22l >. one can 
obtain the recovery bounds of OMP in the noisy scenario. 
In particular, when k = 6 (i.e., s = 25A"), the maximum 
number of iterations becomes 24A, and one can easily check 
that d22l is guaranteed by g^sk < 0.3589, which is better (less 
restrictive) than 5 31K < 0.3333 0. 



In this case, however, the associated computational cost of OMP would 
be substantial due to the large number of least squares (LS) operations. 



IV. Proof of Theorem|3~T 



A. Sketch of Proof 

Our proof is based on the mathematical induction on \T h \ 
(i.e., the number of remaining support indices after k itera- 
tions). First, when |r fe | =0,TC T k so that no more iteration 
is required. Next, we assume that for |T fc | = 1, • • • ,7 — 1, 
if dHJl is satisfied then there exists / < k + [ K L ' ~\ such 
that TCT'. Under this assumption, we will show that when 
|r fc | = 7, if (fl~8T > is satisfied then there exists 



f<k 



4«7 

1 N ' 



(23) 



such that TCT*. 

In the induction step (when |T fc | = 7), we first show that a 
decent amount of support indices in T k can be selected within 
a specified number of additional iterations. To be specific, let 



r«(i 



Er^Di 



L. 



(24) 



where k (> 0) is a multiple of |, then we show that gOMP 
algorithm selects more than 2 L ~ 2 N support indices in T k 
within ki — k additional iterations (i.e., from the (k + 1)- 
th to the fc^-th iteration). In other word, after k^ iterations of 
gOMP, the number of remaining support indices satisfies 



\r k 



<7- 



2 L - 2 N. 







/ i'-th iteration 



N 



N 



(a) The goal of induction step is to show that / < k + ["— ^jy — -] holds 
when|r fc |= 7 (i.e., /<fc + p*£T]). 




(b) The two key ingredients in the proof of / < k - 

\T k L | < 7 - 2 L - 2 Af and 2) / < k L + [M^LL]. 



z'-th iteration 



r^i ■* d 



(25) Fig- 3. Illustration of the induction step. 



Thus, by performing a few more iterations (starting from the 
(kL + l)-th iteration), all the support indices can be selected. 
Specifically, since (f25) directly implies that \T kL \ < 7 — 1, 
by the induction hypothesis it requires at most f-^y — -] 
iterations to select the rest of support indices in T kL Q Hence, 



/ < k L + \ 



4n\r kL 



-]■ 



(26) 



B. Proof of ([25]) 

Before we proceed, we briefly explain the key steps for 



proving (l2~5T >. Consider x r * L and x rfe \ r k 



which are two 



truncated vectors of x r t . From the definition of T k in ([T2l . 

{2 L - 2 N + 
2 L - 2 N and 



pfe 

1 L- 



= {1,2,- •• ,2 L ~ 2 N} and r fe \r£_ a 
1, • • • ,7}, which implies that |r fc \r|_ 
hence an alternative form of (l25t is 



-ll 



The key idea of the induction step is illustrated in Fig. [3] 
We next show that the total number of iterations of gOMP 



is upper bounded by k 
holds true whenever 



f^l . From (|26), it is clear that 



r 4dr fei L , f 4k 7i 



(27) 



Then by using ( |25l >, one can easily show that d23l is guaranteed 
when 



N ' - ' N ' 



(28) 



Now, in proving f < k + \-jp-~\ in the induction step, what 
remains is the verifications of (l2~5l l and d28l >. 



*In the induction step, we have already assumed that when \T k \ < 7 — 1, 



it requires 



maximal iterations to select all indices in T 



jpfcij < |pfc^ r 



i-j 



(29) 



Further, recalling that {xi}i = i,2,-- ,7 are arranged in descend- 
ing order of their magnitudes (i.e., \x\\ > \x%\ > • ■ ■ > 1^7 1)> 
it is clear that x T k\ r k consists of 7 — 2 L ~ 2 N most non- 
significant elements in x r k. Therefore, the sufficient condition 
of (1291 is 



M^t II 2 



< x 



r'\rti 



(30) 



Let B u and B^ denote the upper bound of ||x r 



l-L 



1 2 and the 



lower bound of x 



r fc \rt 



2, respectively. That is, 



s -r fc i. 



< 
> 



B u , 
B t , 



\\Xr*\Tl_J.. 

Then, one can easily check that (130b holds whenever -B u < B^. 
Also, since (f30b is a sufficient condition of (1291 and (l29l 
is equivalent to d25l . one can conclude that (l25l can be 
guaranteed under B u < Bg. 



Let's first find out the upper bound B u of ||x r fc L \\\. Observe After some manipulations, one can easily check that 
that 



Jfeil|2 



= l|y-*x fe -||2 (31) ||r^||2<^||r fc ||2 + ( i + ( 5 7) ^^-|| Xrfcxr ,||2_ (45) 

- ||*(x-x fei )||2 (32) - =1 

> (l-J| TuTfcl ,|)||x-x fcl ||^ (33) By applying Lemma E2 for v = 0, we have 

> (l-VuT^|)H x r^Hi (34) ||r fe ||2 < ||* rfc x rfc ||2 (46) 
where d32i > is due to y = <&x in the noiseless scenario and < (1 + (5 7 )||x r fc ||| (47) 
d33jisfromtheRIP(||x-x^||o<|Tur fe -|). Thus, < (1 + & r )||x r *\ rS ||! (48) 

B u = _' — ■ (35) where d47]l is from the RIP (|r fc | < 7) and gS) uses Tg =0. 

|TuTfcL| Plugging gD into 05) yields 
We next find out a lower bound Bg of ||x r fc* r fc |||. By 

applying Proposition [Z4] to r k \r k -,--- ,r fc + we have ||rfc[|| 2 < (l + 5 7 )^^-'-||x r »\ r *||i (49) 

||r fcl |l2 < C' 1 , fc0 , fcl ||r fc ||2 + (l + ( 5 7 )||x rfcvr ,||i (36) r=o 

||r fc2 ||2 < C , 2, fel ,fc 2 ||r fel ||2 + (1 + ^)11x^^112, (3?) Also, from the definition of I in (E)-©, we have 

: l|xr»\r«.||2</* i " 1 " r Hxr*\r£_ 1 lla ( 5 °) 

\\r kL \\a < CL^^.^Jr^-'Wl + {1 + 6^x^^111, for r = 0, 1, •••, L. Substituting © into ©, we have 



(J8) , (1 + <L) ||x rHr * HI J; 
where we used the facts that v = in the noiseless scenario II r II 2 — /_^ (,WJ 



and that for r = 1, • • • ,L, \T k \T k \ < |r fc | = 7 and hence ^ r=o 

|* rfcvr ,x rfcxr ,+v||2<(l + 5 7 )||x rfc \ r ,||i (39) = ( 1 + ^)ll x rnr|_ 1 lll ^ ( ^ )T 



(h-k^ii-s^^-^ < ( 1 + 'MII*r*\r*_ 1 lll y- (M??r (51) 



From (TlTb and d24t . we have '' 

i = 1, • • • , £. (40) When < jury < 1, we further have 

Since fcj - fa-i > &{*-#■] holds true for i — ].,■•• , ijj we (1 + <L)||x r fc\ r * ||1 

further have ||r fc *||2 < ^ ^_. ( 52 > 

°i,fci-i,fci S exp 1 « j— ^ 1 , i — 1, • • • , L. Thus> me desired lower bound Bg of ||x r ^ rfe ||| is 

(41) 
Since T k C r£ and T^ 1 C T 7 ^ -1 , Tf U T^" 1 C T k L U B /i(l - ^ry)||r^ ||| 

T^- 1 , ir^ UT^- 1 ! < |r| UT^- 1 !, and <5| r fc UTfci -i| < " 1 + <5 7 

(Ji r fc 1 iTfc r -i 1 fori = 1, ••• , X. For notational simplicity, we let . , .. &, , D ^ D . r™ 

l i t u -' ' , ,- As mentioned, it suffices to show B u < Bi to prove ( 1251 1. 

77 = exp ( - k ' r A. U g r I, ~ 1 ' ). then (f36t— (f38T> can be rewritten Using ([35) and ([53), we have 

B » 1 + ^ (54) 

I|r fcl |l2 < r,||r fc ||2 + (l+5 7 )||x rfcxr ,||i (42) B, M (l - W )(l - 5 |TuT , L| ) ' 

l|r fc - Hi < r / ||r fcl ||2 + (l + 5 7 )||x r ^ r ,||i (43) 2 

X2 Since < /xry < 1 and fi(l -jury) = -^{m~ V + ^. by 

I choosing jury = |, /i(l — /iry) takes the maximum value ^- and 

_fei>l|2 ^- M ||«fcn_i||2 1 /1 1 c M|„ .11 2 



| rfc ,||2 < 7? || r — ||^ + (l + ( 5 7 )||x rfcxr ,|^. (44) Bu 4ry(l + <5 7 ) 



ipfc I -D£ 1 — ^ITl lT k L I 

5 From l[T2j, we have fi^i] = 2 r " 1 for r = 1, ■ ■ ■ , £ - 1. As a ' ' 

result, fci-feo = [«(1 + r^l)l = r2«l, fci-fci-l =k+\ K (l+ = A ( 1 + S l) exp (_ R ~ \Ej^ll± Y {55) 

EUi 2T_1 )l -(H[k(1 + E;;\2 t - !)!)= [k-2 1 ] - rK-2 1 - 1 ! for l-0|TUT*i.| V 1 + <5w / 

i = 2, •• • ,£ - 1, and fez, - fe £ _i = [k(2 l " 1 + f-if-Dl - r« • 2 L " fc 1 l- From the monotonicity of isometry constant, we have 

Since k is a multiple of 5, we further have k\ — fco = 2k > n = k[-LjJ-], 

ki-ki-! = K-2 i -K-2*- 1 = K-2 i ~ 1 =«r^lfori = 2,--. ,1,-1, and 7 = \T \ < \T\ = K =*> (5 7 < (5k , (56) 

k L -k L ^= K .2^ + \ K ^]-K.2^=\K\^-]]> K \^-l \Y k L \JT k -- l \<\T\JT k -\ =► V|ut^-|<Vut^|- 

In summary, fc; — fci_i > k[^J-] holds true for i = 1, • • • , L. (57) 



Using d56l l and (|57] >, (l55T l can be rewritten as 
■ exp — k 



Bu 4(1 + S K ) __ ( J-- $\TuT k L | 

B f ~ 1-5, 



d \TUT k L\ 



1 



>iV 



From ( 1581 1. it is clear that B„ < £><> under 



4(1 + 5 K ) 



1 -< 
Equivalently, 



>\TUT k L\ 



exp 



1 



- , |TUT fc i| 



1 + Jw 



< 1. 



l + 5 N . 4(1 + S K ) . 

— In < K. 



1 0|TuT fc -L| 1 — "\TuT k L\ 



(58) 



(59) 



(60) 



We now investigate the order \T U T L \ of the isometry 



constant 5 



\TUT k L\ 



in ( f60b - First, we observe that 



nhr 



\TUT KL \ = \T\T kL \ + \T kL \ 
= \T kL | + \T kL | 
< 7 - 2 L - 2 N + k L N, (61) 

where (ED is from d25j. Noting that \T U T fcz *| and 7 - 
2 L ~ 2 N + fc^A*" are integers, we have 

\T(JT kL \ < j-2 L ~ 2 N + k L N -1 (62) 

< 7-^\T k L \ + k L N-l (63) 



< K + fN-1 

1 JV ' 



< K 



AkK 



+ 1JJV-1 



(64) 
(65) 

(66) 
(67) 



= (4k+1)A" + N-l, 

where ( |63l is from Lemma 12.11 ( |64l is due to 7 = |T | < 
\T\ = K and k L < f (see Fig. [3(b)| >, and $65$ follows from 
the fact that / < \^jf]. Denoting s = (4k + 1)K + N - 1 
and using (l60l l in doTl i. one can show that (|25T > holds under 

1 + 5 N . 4(1 + 6k) . 

- In — < k. 



I 



1 



In this subsection, we show that k^ + [ ^ 7 N -] < 



C. Proof of 

In this su 
k + \^-~\ . First, from the definition of k^, 



A^ 1 N 

T = l 



(68) 



i-i 



*+r«( i +Er 1 jf L i)+«r ^n (69) 

r— 1 



L-l 



= fc + [k(1 + ^ 2 r ~ 1 ) + 4 

T = l 

= k+ \k-2 l ~ 1 +k\- 



N 



N 



(70) 



(71) 



where d2Qj is from the fact that \T k \ = 2 T ~ l N for r = 
1, • • • ,L - 1. Noting that |r*| < 2 L ~ X N (LemmaEB and 
is a multiple of \, we have 

fc L < k+\n-2 L ~ 1 



K-2 



L-li 



k + K-2 1 



(72) 



Then it follows that 






= k + k ■ 2 L 4 

- * + 0> 

which completes the proof of 



N 

r#i-~ * 



(73) 



A s-upp(u) 



|r fc || 2 



V. Proof of Theorem 13. 3 1 

A. Sketch Proof of (Qjj) 

The proof of ( [T9T > is similar to that of Theorem 13.11 in 
the sense that it is based on the mathematical induction 
of the number of remaining indices |r fc | after k iterations. 
We first consider the case when |r fc | = 0. In this case, 
all support indices are selected and T C T k . Since x fc = 
'|y — «&u|| 2 , we have 

= lly-*x fc || 2 

= min ||y — 3?u||2 

supp(u)—T k 

< ||y-*x|| 2 

= ||v|| 2 . (74) 

Next, we assume that for |r fc | = 1,2, • • • ,7 - 1, if (ED is 
satisfied then there exists f < k+ \ N ~\ such that ||r^||2 < 
C"||v||2. Under this assumption, we show that when |r fc | =7, 
if (Bil l is satisfied then there exists f < k + \^ff-~\ such that 
HH2 < C||v|| 2 . 

When |r fe | = 7, we show that a decent amount of indices 
in r fe is selected within a specified number of additional 
iterations. Specifically, we show that gOMP algorithm selects 
more than 2 L ~ 2 N support indices in T k within kL — k 
additional iterations (kL is defined in (f24b). In other words, the 
number of remaining support indices \T kh \ after k^ iterations 
of gOMP satisfies 

|r fcL | < 1 -2 L ~ 2 N. (75) 

Since \T kL | < 7 — 1, it is clear from the induction hypothesis 
that there exists f < k L + [" ly 1 sucn that ||r^|| 2 < 



C"||v||2. In other words, within k^ + {-js — ~\ i terat i° ns ^ tne 
£ 2 -norm of gOMP residual fails below a constant multiple 
of the noise energy. Since the ^ 2 -norm of the residual is a 
deceasing function of the iteration number (||r 4 || 2 < 1 1 r- 3 1 1 2 for 
i > j)> if we show that 



k L + \ 



4K|r fc 

7V~ 



h<* + r^i, 



(76) 



then we must have / < k + \-jr-~\ and the induction step is 
completed. In fact, using d75l ) and ( |76l ), one can easily show 
that ( T76b holds whenever 

h + r ^-^' i^+ffi, 

Since we already proved this in ((73}, we can conclude that ( T76| ) 
holds true and hence the induction step is completed. Now, 



what remains in the induction step (when |r fe | = 7) is the also noting that Fq = 0, (|82T > becomes 
proof of (|75l l, which will be presented in the next subsection. L 

\\v kL \\l < ^77 L " r ||*rnr-x rn r-+v||2 (83) 

T = 

B. Proof of CHl L 

< ^27 ? L ^(||* r , vr ,x rfc \ r ,||2 + ||v||2)(84) 

The proof of (1751 1 is similar to the proof of (125V Instead of r=0 
directly checking ( [75) . we show that the sufficient condition L 

of d75l l is true. To be specific, we show thaO < 2(1 + <5 7 ) \~] 7 ? L_r || x r fc \r fc 



2 
2 

T = 



K*x;||2<l|xr*\r*_Jl2- (77) 



2E^ r H v ll2 (85) 



To do so, we first construct lower and upper bounds of || r kL || 2 L 

and then derive a condition guaranteeing (fTTV , 0(1 _l A ") V^ n L ~ T \\-x 

First, we build a lower bound of ||r' c - L ||2. Noting that ||x — T=0 

x fci || < \TUT kL \, we have 2||vll 2 



2 



|r^|| 2 = ||y-*x fci 



l-n 



(86) 



where §5} uses the RIP (|r fe \i^| < |r fc | = 7 for r = 

= ||*(x-x^)+v|| 2 "— ' -' 1U '^ 

> ||*(x-x fc M|| 2 -||v|| 9 0,l,---,L)and®isduetoj7 = exp(-K ^ )< 

_ " ,- " " 1 and hence V L n L - T = T L rf < V°° n L - T = -±- 

> /T^5 l! - - - kh M ~ - H" 11 - ^ r= 9 (. . . ^ r =9 'r^ A4^r° ' . !-»?' 



| TuTfcL 1 ||x - x || 2 - ||V|| 2 Als0; from ^ definition of L in (fl3T>— dTST>. we have 

■^1- W^I|x r , L || 2 -||v|| 2 . (78) l|xr*\r«.Hl < A* i_1 ~ T ||xr*\r*_Jla (« 7 ) 

for r = 0, 1, • • • , L. Substituting d87l > into d86*l l. we have 
Next, we find out an upper bound of ||r fcl -|||. By applying 2 , 

Proposition El we have " " || r fe ^|| 2 < r ' tXr *- 1 V (un) L ~ r I 2 ^ V ^ 

||r fcl |l2 < C'i,*o,fc 1 ||'*lla + ll*r*\rf*r*\i*+v||», (79) (88) 

llr fe2 ll 2 < To, , llr fcl ll 2 + ll<J> „, ,x , , + vll 2 (8f» We ch °° Se W = 2 SO that Er=o(w) L T = Er=oF < 
||r || 2 5= <^2,fc 1 ,fc 2 |l r II2 + ll v r fc \rj x r fc \rj + v lbj vouj y^oo J_ _ 2 and 

Jc f .||2 ^ Q„/ 1 , C N ||„ II 2 , 2 ll v ll2 



•^ > r ^ „..a ;-"« inri x 1 * Xri+ v||5. II^II^S^l + ^Hx^^JI^^-M. (89) 



: C L , kL _ ukL \\r K ^\\i + \\^ r ^ rkr K rkXrhr +v|| 

(81) Hence, the upper bound of \\r kL || 2 is 



Using (ED, we further have ||r fel || 2 < 2J277 (1 + <0|x rnr * || 2 + ||v|| 2 . (90) 

V \ L-l ^/l _ ^ 

||r fcl || 2 < ?7||r fe ||2 + ||$ r fcx r fcX r fcN r fc + v|| 2 , Now, using the lower bound in (T78T> and the upper bound 

1 2 in d90l ). we have 



< ^|r Bl |l2 + ll*r*\r* x r*\r*+ v 



r^|| 2 < r 7 ||r fci - 1 ||| + ||* r i.v r *x rnr » +v|| 2 , 



||x r * i || 2 <a||x r *\ r »_ i || a + / 8||v|| a (91) 

where 



2r/(l + 5- t ) a _ 72 + yr^ 



a = 2, — —. ^- and 8 



where 77 = exp ( - k l^^f; '\ y After some manipula . V \tut"l \ ^/(l - ?y)(l - <S| TuTfcL ,) 



i+<5 
tions, we have 



Recalling that 77 = cxp ( — k ^+5 )> one can snow 

that a < 1 is guaranteed by 



\r^\\l< V L \\r k \\l + Y,V L - T \\*r*\r^\ n +v\\l (82) / 

^ 8(l + 6» z 7 ^l-<5|r*uTH-i| 



1 exp ~ K 7±! - L (92) 

By applying Lemma l2~2l (i.e.. |ir fe || § < ||# r fcx r fc +v||n) and . . .„ „> , 

j- ffj- s 1 — iv ,11 112 - 11 1 r 112^ Further, noting that 5| r | uTfcL -i| < (5| TuTfcl ,| and |TUT fei | < 

(4k+1)K + N-1 (see (Ell), and denoting s = (4k + l)i^ + 

6 Recalling that {ii}j = i,2 ( ... l7 are arranged in descending order of their 7y 1 <- -1 KQlflc true if 

magnitudes (|xi| > IX2I > ■ •• > |^t|)> it is clear that x rfc , rfc consists 

\ L-i o/'-i 1 x \ f 1 r 

of 7 — 2 L ~ 2 N most non-significant elements in x F k, and therefore J77t is ' °i<"7 1 



p fc , ana tneretore yn) is ^ " ; ex p n < J (93) 

a sufficient condition of j75t . 1 — 5 S \ 1 + r5jv 



Equivalently, 



we further have 



1 + 6 N , 8(1 + 6k) . 

=— In < K. 

1-S. 1-6. ~ 



(94) 



We consider the case when /3||v|| 2 < (1 — a)||x rfc \ r fc ||2. In 
this case, d9TT i implies that 

II 1 1 2 , || 1 1 2 

Il x r^ll2 < llxr^vr^jh) 

and hence d75l > holds true. The alternative case (i.e., /3||v|| 2 > 
(1 — a)||x r fcx r fc || 2 ) is trivial. Indeed, when /3||v|| 2 > (1 — 
a)||x r fc\ r fc || 2 , (l90t implies that 

\\r kL h < (i^^l-^lrur^i-ljllvlla 

a + B-1 

< — ; v 2 

1 — a 

< C||v|| 2 . 



C. Proof of (|20j 

So far we have shown that ([T9T > holds true under (I2H and 
we are now ready to prove (|20V First, we observe that 

|| r-^ || 2 = ||y-*x / || 2 

= H^x-x-O + vJI;, 

> ||*(x-x^)|| 2 -||v|| 2 

> (1-*|tut/|)||x-x'|| 2 -||v|| 2 (95) 

where J95] l is due to the RIP. We now consider the order |TU 
T*\ of the isometry constant 6\t\jti\- Note that 

\TUT f \ = \T\T f \ + \T f \ 
ik 



:■ |r\r'-'-|+jyrr^l 



(96) 
(97) 



< \T k \-2 L - 2 N + N\— — 1, (98) 



where d97| > is because k L < f and / < |~^p~|, and 
due to (F75t . Since |T U T*\ is an integer, we have 



is 



4re&f 

ITUT-^I < |T fc | — 2 i ^ 2 A^ + A^T 1-1 (99) 

1 AkK 

< \r k \-^\r h L \ + N\^-i (ioo) 



< K + ^^-i 

1 N ' 

< * + *(^ + l)-l 



(101) 

(102) 
(103) 



= (4k + 1)K + N-1, 

where (flOOl is from LemmaOand ([Toll is due to |r fe | < X. 
Let s = (4k + 1)/4T H- TV — 1, then using d95l ), we have 

||r / || 2 >(l-^)||x-x/|| 2 -||v|| 2 . (104) 

Equivalently, 



x / || 2 < 



|gj|2+ ||V||2 

1-8. 



(105) 



Recalling from (fl~9b that 



ll»-^ll,< (1+ i y =C||v||„ (106) 

which completes the proof of Theorem 



VI. Conclusion 

With the aim to enhance the recovery performance of OMP, 
gOMP algorithm has been proposed. Unlike OMP selecting 
one index in each iteration, gOMP chooses multiple indices, 
which helps to catch more than one support indices in each 
iteration. While gOMP has been shown to be competitive 
empirically, little has been known whether it can perform a 
uniform recovery of sparse signals under a natural RIP. In this 
paper, we have shown that gOMP performs an exact recovery 
of any /^-sparse signal in noiseless scenario, provided that 
87K+N-1 < 0.0231. We have also shown that in the noisy 
scenario gOMP can provide a stable recovery of any if-sparse 
signal from its perturbed measurements under 8hk+n-i < 
0.0627. It is worth mentioning that the stability result can 
be readily extended to the more general scenario where the 
underlying signals to be recovered are approximately sparse. 
Also, the presented proof strategy might be useful to obtain an 
improved recovery bound of greedy algorithms derived from 
the OMP. 

Appendix A 
Proof of Lemma|2~2"1 



(107) 



Proof: T'nTC T k implies that 

l|r fc ||^||P^y||^<||P^ nT y||^. 



Also, noting that P^ fc T y is the projection of y onto the 
orthogonal complement of span($> T k nT ), 

||P£* nr y||!= min ||y-fe||*. (108) 

supp(z.)=T k nT 

From ( 1107b and ( 1108b . we have 

||r k ||a < ||y-* Tfc nTX T fcnT|| 2 

= ||*txt + v- * T <c nT x TfcnT || 2 

= ||* rfc x r , +v||2, (109) 

where (fT09l is from T\(T k n T) = T\T k = T k . ■ 



Appendix B 
Proof of Proposition ^. 31 

The following lemmas are useful in the proof of Proposition 



Lemma B.l: Let u, z £ lZ n be two distinct vectors and let 
W = supp(u) n supp(z). Also, let U be the set of N indices 
corresponding to TV most significant elements in u. Then for 
any integer N > 1, 



|W|. 



(u,z)< A /r^niu c/ || 2 ||z w || 2 . 



(110) 



f h<c 



r J 2 



v 2, 



Proof: We first consider the case 1 < N < \W\. Without 
loss of generality, we assume that W = {1, 2, • • • , |W|} and 



10 




£ 2 -norm columns. Then, for any integer I > k, the residual of 
gOMP satisfies 



-r ;+1 ||2> 



1 



l*A*+ir' 



'||2 



1 + ^..-AW-iMl2- 



(119) 



Fig. 4. Illustration of indices in Wi . 



w m 



w 



that the elements of \iw are arranged in a descending order 
of their magnitudes. We define the subset Wi of W a£| 



*-\{N(m-i)+i,--,\w\} i =m 



rffli-i 



Then it is clear that 

(u,z) 






< 



£u» 



Wil|2||ZwJ|2) 



(HI) 
(112) 

(113) 
(114) 



Proof: Recall that gOMP algorithm orthogonalizes the 
measurements y against previously chosen columns of 4>, 
yielding the updated residual in each iteration. That is, 

r ;+1 = P^ i+1 y. (120) 

Since r' = y — <I>x', we have 

r' +1 = P^ fc+1 (r ; + *x ; ) = P^ i+1 r'. (121) 

where ( 11211 > is because <£>x' £ span(& T i) and T l C T t+1 
and hence P^, +1 <&x z = 0. As a result, 

r i _ r i+i = Y i _ P ± l+ y = p T , +ir '. (122) 

Noting that A /+1 C T l+1 , we have 

\\v l - v l+l \\ 2 = ||P Ti+ ir'|| 2 > llP^+ir'lU. (123) 

Since P A !+i = P A , +1 = (* A j+i)'* A t+i> we further have 



\v l -v l+1 \ 



2 > ||(* A!+1 )'* Ai+I r'|| 2 

"* Al+1 r'|| 2 (124) 



> 



where ( 1113b is due to the Holder's inequality. By the definition 
of U, we have ||ur/|| 2 > Hu^J^ = max.; ||u^|| 2 and hence 



< 



Mh£] 



z wA\2 



\W\. 



< \\nuhM^]^2\\z m 



(115) 
(116) 



\M^ Uu \\ 2 \\z w \\ 2 



VI + S N 

where (1124b is due to the fact that the singular values of ^^i+i 
lie between VI — $n and vT+^Jv- ■ 

In the next lemma, we find out a lower bound of 

W A , + y\\2. 

Lemma B.3: Let x £ lZ n be any A"-sparse vector supported 
on T and $ £ R mXn be the measurement matrix with unit t 2 - 
norm columns. Then, for a given set T k and an integer I > k, 
the residual of gOMP satisfies 

1 - <5|r*uT'| 



(117) ||* AI+ ir'||^> 



I w I 



where (|1161 l follows from the fact that ^Z i=1 a-i < 



|z Wi || 2 andd=r J ^ L l. 



dJ2i=i a l witn a 

Next, we consider the alternative case where N > \W\. In 

this case, it is clear that J \'-j^- ] = 1 and ||ur/||2 > ||ujy|| 2 , 
and hence 



(||r'|||-||#r*\r*xr*\r*+v||l) 

(125) 

Proof: Let z £ lZ n be the vector satisfying z TnT k ur k = 
x TnT fc ur fc and Zm(TnT fc ur fc ) = 0. Further, let u = 3>'r', 
U = A l +\ and W = T*\f l E Then using Lemma EQ we 
have 



(*'r',z) 



< 



l^WuuHzwh = || UC7 || 2 || ZW || 2 

> ||U W ||2||ZW||2 

> (Uff,Zff) 

= (u,z>, (118) 



which is the desired result. ■ 

The following lemma characterizes the reduction of residual 
(in £ 2 -norm) in the (I + l)-th iteration of gOMP 

Lemma B.2: Let x £ H n be any A-sparse vector supported 
on T and 3? £ 'R. mxn be the measurement matrix with unit 



r*\r'| 



N 



ll|* AI+ ir'|| 2 ||z r;NT1 || 2 



: yr^lll*A«+xr'lla||ar*\r«l|a- (126) 

Since supp(x. 1 ) = T l and supp(&'r l ) = f2\T, we have 
(<&'r',x') = and (<&V,z) = ($'r',z — x') and thus 

(*'r',z> 



ll*A^ r i2 > 



r^niz rnT1 || 2 

(*V,z-x*) 

r^niz rnT1 || 2 



(127) 



7 We note that when \-jj- 
elements. 



1 > LnJ- , the last set W \w\-. has less than TV 



8 [7 = A !+1 is because A' +1 contains the indices corresponding to N 
most significant elements in <i?'r'. Since supp(u) = Q\T l and supp(z) = 
T n T k U T* and also noting that T k C T l , we have VK = supp(u) n 
supp(z) = F k .\T l . 
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Furthermore, we have 






*y, z -x') 






= (*(z-x'),r') 




(128) 


= I(||«_(_-x')||_ + 


llr'Ul- 


||r i -*(z-x , )||_)(129) 


= ±(||*(z-x')||_ + 


llr'Ul- 


||*(x-z)+v|||) (130) 


= I(||*( z -x I )||i + 


\\A\l- 


||* r fc\ r fcX r fc\ r «, + v|||) , 
(131) 



where (II 301 > uses the fact that r + <I?x = y = <I?x + v. 
In proving ( 11251 >, we consider the following two cases. 
• First, if ||r'|||- || *r fc \r^ x r fc \r^ + v lll < 0> G23 is true 



since H^+ir 

_/||2 



2 II 2 



> 0. 



Next, if || x* 11 2 — ||*r fc \r fcX r fc \r fc + v lli2 — 0' tnen using 
inequality _(a + b) > y/ab (with a = |<J?(z — x z )||| and 



6 = ||r ( || 2 — ||*r fc \r fcX r fc \r fc + v lll)> <l 1 3 lb becomes 

(_»V,z-x') 

> ||*(z-x ( ) 



'"'III- ||*r*\rjx rfc \ r j + v|||. 
(132) 
Noting that z — x' is supported on r£ U T, we have 

> J -<*|r*ur'| l|z-x'|| 2 



|*(z- 



,h 



where (I133l l is due to (x. l ) n \ T i 
( 1133b into (11271 . we have 



- V 1 ~~ VjUT'l ll( z - x )n\T'l|2 

= J 1 - VrUT'l ll z n\T ! l|2, (133) 

0. Plugging ( fT32b and 



l*A«+^lla 



> 




<Ppfc\ pA; Xpfc\ pfc + V 



Combining these two cases, we obtain the desired result. ■ 
We are now ready to prove Proposition 12.31 

Proof: Using Lemma IB. 21 and IB. 31 and also noting that 



r r 



r U - r 



= ||r 

l+l\\2 



2 112 



— ||r i+1 ||n, we have 



1-5 



> 



l ^ UT ' 1 "(llr'HS 



(1+^)1^1 



|*r fc \r* x r*\rj + v|| 2 ) , 
(134) 



which completes the proof. 



Appendix C 
Proof of Proposition ^. 41 

Proof: Subtracting both sides of (fTTT i by ||r J | 
|$ T \pfcX T \pfc |||, we have 



r + l_-|l*r*\r**r*\r*+v||_ 



< 



1- 



1-* 



|pfc UT i| 



[•^1(1 + * 



N 



x(\\r l \\l 



\& r k\ T kX r k\ r k+v\\l). (135) 



Since 



1 



l-5„ 



> 0, we have 



1 _ ^|r*uT<| 
[^1(1 + 5 N ) 



< exp 



)|pfc UT i| 



^1(1 + *. 



N 



Hence, 



J + l||2 



- 11*1".' 



pfe\rfeXpfc\rfc + vy 2 



< exp 



1-5 



|pfc UT i| 



r^i(i+<5 w ) 




/||2 



l*r*\r£x r fc\ r fc + v ll2), 



and also 



J+ 2 II 2 



< exp 



I* 



|*r fc \r£ x r fc \r£ + v|| 2 
/ l-J|rfcuT'+i| \ ,, 

r fc\ r fcXpfc\pfc + v|| 2 ), 



n' + l||2 



J' II 2 



$pfcypfcXpfc\rfc + v|| 

/ 1-5,, 



ex P i- rlrfc '7 uT '; 1 ; (n/- 1 ^ 

V \ l -^](l + 5 N )J 
-||*rfc\r* x r*\r* -+- v|||). 
After some manipulations, we obtain 

Il r ''ll2 - ll*r*\r*x r »,\r* + v||_ 
i'-i 



< 



1 



J|pfe uT !| 



n° x p 

■i=i \ i ~iv~ 
x(l|r'||2-||*r*\rjx r *\r5+v||l). 



f^Kl + M, 



Since C T ,i.y > 0, it is clear from (??) that 



J'||2 



||r ||a < Cr,M'||r'||| + ||*r fc \r^x rfcX r, + v 
which completes the proof. 



l2> 



(136) 



(137) 



(138) 



(139) 



(140) 
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